The invention relates to anomaly detection in a computer and telecommunication networks in which the concept of normal behaviour varies with time. More particularly, the invention relates especially to teaching an anomaly detection mechanism. An example of such an anomaly detection mechanism is based on self-organizing maps (SOM).
Mechanisms for detecting abnormal situations belong to one of two major categories, namely rule-based detection mechanisms and anomaly detection mechanisms. Rule-based detection mechanisms attempt to recognize certain behaviour patterns which are known to be improper. Thus, rule-based detection mechanisms have two severe limitations: they can only detect problems which have occurred before and which have been explicitly taught to the detection system or programmed into it. Anomaly detection systems (ADS), as used in this application, reverse the detection problem: they are taught what normal behaviour is, and anything deviating significantly (by a predetermined margin) from the norm is considered anomalous. ADS mechanisms are capable of detecting potentially problematic situations without explicit training of such situations. An example of an ADS is disclosed in reference 1. Thus an ADS is defined as a mechanism which is trained with normal behaviour of the target system. Accordingly, an ADS flags every significant deviation from normal as a potential anomaly. In contrast, a rule-based detection system is trained with known modes of abnormal behaviour and it can only detect the problems that have been taught to it.
Reference 1 discloses an ADS for a Unix-based computer system. The system consists of a data-gathering component, a user-behaviour visualization component, an automatic anomaly detection component and a user interface. The system reduces the amount of data necessary for anomaly detection by selecting a set of features which characterizes user behaviour in the system. The automatic anomaly detection component approximates users' daily profiles with self-organizing maps (SOM), originally created by Teuvo Kohonen. A crucial parameter of an SOM is a Best Mapping Unit (BMU) distance. The BMUs of the SOMs are used to detect deviations from the daily profiles. A measure of such deviations is expressed as an anomaly P-value. According to reference 1, the ADS has been tested and found capable of detecting a wide range of anomalous behaviour.
A problem with known SOM-based ADS mechanisms is that they are restricted to detecting problems in systems having a well-defined normal behaviour. In most telecommunication networks the concept of “normal behaviour” is, at best, vague. A network element's behaviour at peak time is very different from its behaviour at the quiet hours just before dawn. More precisely, most often it is the users who cause the variation in what is called normal. In other words, known ADS mechanisms do not readily lend themselves to detecting problems in systems or elements whose normal behaviour varies with time.